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Abstract — The regularized random forest (RRF) has recently been proposed for feature selection by building only one ensemble. 
However, in RRF the features are evaluated by a part of the training data at each tree node, and thus the feature selection process 
may not be stable. Here an enhanced RRF, referred to as guided RRF (GRRF), is proposed. In GRRF, the importance scores from 
an ordinary random forest (RF) are used to guide the feature selection process in RRF. Experimental studies show that GRRF, in 
general, is able to select compact feature sub sets and is better than RRF, varSelRF and LASSO logistic regression in terms of the 
accuracy of RF and a decision tree method C4.5. Both RRF and GRRF were implemented in the "RRF" package available at CRAN 
I http://cran.r-project.org/ 1, the official R package archive. 

Index Terms — attribute selection; dimension reduction; feature selection; variable selection; classification. 



1 Introduction 

Given a training data set consisting of N instances, 
P predictor variables /features Xi,i = l,...,P and the 
class Y <s {1,2, ...,C}, the objective of feature selection 
is to select a compact variable /feature subset without 
loss of predictive information about Y. Note feature 
selection selects a sub set of the original feature set, 
and therefore may be more interpretable than feature 
extraction (e.g. principle component analysis [1] and 
partial least squares regression |2|) which creates new 
features based on transformations or combinations of 
the original feature set (3). Feature selection has been 
widely used in many applications [4] as it can moderate 
the curse of dimensionality, improve interpretability |5| 
and avoid unnecessary work analyzing irrelevant and 
redundant features. 

Information-theoretic measures such as symmetrical 
uncertainty and mutual information have been success- 
fully used for feature selection, e.g., CFS (correlation- 
based feature selection) |6) and FCBF (fast correlation- 
based filter) [7J. However, these methods have difficul- 
ties in capturing high-order interactions between vari- 
ables. One difficult example for these methods is the 
exclusive OR relationship: Y = XOR(Xi, X2), in which 
neither X\ nor X2 individually is predictive, but X\ and 
X2 together can correctly determine Y (SJ. 

Classifiers LASSO logistic regression J9) and linear 
SVM (SVM-RFE HQ)) are well-known feature selection 
methods. However, these methods assume a linear re- 
lationship between the log odds of the class and the 
predictor variables (LASSO logistic regression) or be- 
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tween the class and the predictor variables (linear SVM). 
Furthermore, before using these methods, one often need 
to preprocess the data such as transforming category 
variables to binary variables and normalizing variables 
of different scales. 

A powerful classifier called random forest (RF) IITTl 
has been commonly used for measuring feature im- 
portance f!2l , as it naturally handles numerical and 
categorical variables, different scales, interactions and 
nonlinearities, etc. Though the RF feature importance 
scores can be used to select A' features with the highest 
importance scores (the K best features), there could be 
redundancy among the K variables, which is different 
from feature selection that selects a set of relevant but 
non-redundant features (the best A'-feature sub set). 
Similarly, Boruta 1131 . a method based on RF, aims at 
selecting a set of relevant features but does not consider 
the redundancy between the features. 

varSelRF Q, a feature selection method based on 
random forest, has become popular. varSelRF consists of 
multiple iterations and eliminates the feature(s) with the 
least importance score(s) at each iteration. Since elimi- 
nating one feature at each iteration would be computa- 
tionally expensive, the authors considered eliminating 
a fraction, e.g., 1/5, of the features at each iteration. 
However, the resolution can be too low when the size 
of the best feature set is large. 

Recently, the regularized random forest (RRF) has 
been proposed for feature selection by building one 
ensemble [14), instead of building multiple ensembles 
ID, Il5l . However, in RRF the features are evaluated by 
a part of the training data at each tree node, the feature 
selection process may not be stable. 

Here we propose a guided RRF (GRRF), in which the 
importance scores from an ordinary random forest are 
used to guide the feature selection process in RRF. Since 
the importance scores from an RF are calculated based 
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on all trees in the RF and all the training data, GRRF 
is expected to perform better than RRF. We used 10 
gene expression data sets for our experiments and the 
results show that GRRF can select compact feature sub 
sets, and the accuracy performance is better than RRF, 
varSelRF and LASSO logistic regression when a decision 
tree algorithm C4.5 (16| or RF is used as the classifier. 

Section Background presents previous work. Section 
Guided Regularized Random Forest describes the GRRF 
method. Section Parameter Selection introduces the pro- 
cedures of parameter selection. Section Experiments 
presents and discusses the experimental results. Section 
Conclusion concludes this work. 



2 Background 

2.1 Variable importance scores from Random Forest 

Random forest (RF) [ l l J is a supervised learner that 
consists of multiple decision trees, each of which grown 
on a bootstrap sample from the original training data. 
The Gini index at node v, Gini(v), is defined as: 



Algorithm 1: Feature selection at node v. 

input : F and A 
output: F 

1 gain* 4— 0, count 4— 0, X* -i— null; 

2 for m <— 1 to M do 

gain^^Xm , v) 0; 

if X m £ F then // calculate the gaina for all 
variables in F 

gainn(X m: v) <— gain{X m .v) 

end 

if X m ^ F and count < \\/~M^ then 
gain^{X m ) <— A ■ gain(X rn ) ; 
count <— count + 1; 

end 

if gainn (X^ , v) > gain* then 

gain* <— gainn(X rn . v), X* 4— X m ; 

end 

14 end 

15 if X* ^ null and X* ^ F then 

16 | F <- {_F, X*}; 

17 end 

is return F 



the main difference is that the regularized information 
gain, Gainn(Xi,v), is used in RRF: 
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k=l 

where p% is the proportion of class-fc observations at 
node v. The Gini information gain of Xi at node v, 
Gain(Xi,v), is the difference between the impurity at 
the node v and the weighted average of impurities at 
each child node of v. 

Gain(Xi,v) = Gini(Xi,v)—WLGini(Xi,v L )—wnGini(Xi, 

where v L and v R are, respectively, the left and right 
child nodes of v; u>l and wr are, respectively, the ratios 
of the number of instances at the left and right child 
nodes to the number of instances at node v. At each 
node, a random set of mtry features out of P is evalu- 
ated, and the feature with the maximum Gain(Xi,v) is 
used for splitting the node v. 

The importance score for variable X; can be calculated 
as: 
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where F is the set of features used for splitting in 
previous nodes and is an empty set at the root node 
in the first tree. A £ (0, 1] is called the coefficient. The 
coefficient is used to penalize using a feature Xi ^ F 
for splitting node v. A smaller A leads to a larger 
penalty. RRF uses Gainn(Xi,v) at each node, and adds 
a new feature to F if the feature adds enough predictive 
^fiiriformation to the selected features. Figure [T] illustrates 
the feature selection process with an example. The nodes 
in the forest are visited sequentially (from the left to the 
right and from the top to the bottom). Three distinct 
features used for splitting are added to F in Tree 1, and 
one feature X 5 is added to F in Tree 2. Algorithm [l] 
shows the feature selection process at each node. 

Similar to Gainn(-), a penalized information gain was 
used to suppress spurious interaction effects in the rules 
extracted from tree models [17|. The objective was differ- 
ent from the goal of a compact feature sub set here. Also, 
the regularization in [17 \ only reduces the redundancy 
in each path from the root node to a leaf node, but 
the features extracted from such tree models can still be 
redundant. 



where Sxi is the set of nodes split by Xi in the RF with 
ntree trees. The RF importance scores can be used to 
select the K features with the largest importance scores 
(the K best features), but they can not be used directly 
for feature selection which aims to select the best K- 
feature sub set. 



2.2 Regularized Random Forest 

The regularized random forest (RRF) [14) applies the tree 
regularization framework to RF and can select a compact 
feature sub set. While RRF is built in a way similar to RF, 



3 Guided Regularized Random Forest 

In RRF, F starts from an empty feature set, and the 
features entering F early have a large effect on the 
following feature selection process. Also, the features are 
evaluated at each node with only a part of the data, and 
therefore the feature selection process may not be stable. 
Here we propose the guided RRF (GRRF), which uses the 
importance scores from a preliminary RF to guide the 
feature selection process of RRF. Since the importance 
scores from an RF are calculated based on all trees in 



3 



F={}+X1 F={X1 ,X3,X4} 




Tree 1 Tree 2 



Fig. 1. The feature selection procedure of RRF. The non-leaf nodes are marked with the splitting variables. Three 
distinct features used for splitting are added to F in Tree 1 , and one feature X 5 is added to F in Tree 2. 



the RF and all the training data, GRRF is expected to 
perform better than RRF. 

Note Impi is the importance score of variable Xi 
calculated from an RF (Equation (]}). To normalize the 
scores between and 1, a normalized importance score 
is considered: 



Imp'i 



Impi 



max^ =l Impj 



(3) 



Instead of assigning the same coefficient to all features 
in RRF, GRRF assigns a coefficient to each feature, that 
is, 

' \i ■ Gain(Xi,v) Xi ^ F 
Gain(Xi, v) Xi G F 



Gain R (Xi,v) 



(4) 



Xi G (0, 1] is the coefficient for Xi (i G {1, —,P}) and 
is calculated based on the importance scores from an 
ordinary RF: 



\i = (1 - 7) * A + 7 * Imp'i 



(5) 



where Ao G (0, 1] controls the degree of regularization 
and is called the base coefficient, and 7 G [0, 1] controls 
the weight of the normalized importance score and is 
called the importance coefficient. Note RRF is a special 
case of GRRF with 7 = 0. Given Ao and 7, a feature with 
a larger importance score has a larger Aj, and therefore 
is penalized less. 

In our experiments we found the feature size can be 
effectively controlled by changing either Ao or 7, but 
the latter often leads to better performance in terms of 
classification accuracy, and therefore we consider 7 as 
the only parameter for GRRF with a fixed Ao = 1. Since 
Ao = 1, Aj = (1 - 7) + 7 * Imp'i = 1 - 7(1 - Imp'i). Since 
1 Imp'i - 0' a larger 7 leads to a larger penalty on 
feature Xi that does not have the maximum importance 
score, i.e., Imp' l ^ 1. 

4 Parameter Selection 

Cross-validation (CV) can be used to optimize the pa- 
rameters. For each parameter setting, the CV error can 
be calculated as follows. 

(1) The training set D is partitioned into M sub sets 
of equal (or approximately equal) sizes (called M-fold 
CV). Let Di be the i th sub set and D_ i be the set formed 
by the remaining sub sets, i.e. D_i = {D — D,}. 



(2) Apply the feature selection method with the 
parameter setting on _D_i and select a feature sub set. 

(3) Build a classifier on the using the selected 
feature sub set. 

(4) Apply the classifier on the testing fold Di. Let errt 
denote the testing error rate. 

(5) Repeat step (2) to (4) for i = 1, ...M and then 
calculate the CV error as Ylj=i errj/M. 

The parameter setting with the minimum CV error 
may be selected. Let getBestPara(D,AI) denote the 
procedure selecting the best parameter setting, given 
D, the training data and M, the desired number of folds. 

In many cases, CV is also used to evaluate the perfor- 
mance of a model. The procedure, referred to as nested 
cross-validation, is described as follows. 

(1) The training set D is partitioned into Mi sub sets 
of equal (or approximately equal) sizes. Let Di denote 
the i th sub set and = {D — Di}. 

(2) Select the best parameter setting using getBest- 
Para(D_i,M 2 ). Note M 2 can be different from M x . 

(3) Apply the feature selection method with the se- 
lected parameter setting on Di and select a feature sub 
set. 

(4) Build a classifier on D-i with the selected feature 
sub set. 

(5) Apply the classifier on the testing fold Di. Let erri 
denote the testing error rate. 

(6) Repeat step (2) to (5) for i = 1, ...Mi and then 
calculate the CV error as Y^j=i err j /Mi. 



5 Experiments 

10 gene expression data sets used in |4] were considered 
in our experiments. The data sets are summarized in 
Table[T] We implemented the RRF and GRRF algorithms 
in the "RRF" R package based on the "randomForest" 
package (27|. The "RRF" package is freely distributed 
under GNU General Public License and is available from 
CRAN (http://cran.r-project.org/), the official R pack- 
age archive. We also evaluated two well-known feature 
selection methods: varSelRF available in the "varSelRF" 
R package and LASSO logistic regression available in 
the "glmnet" R package. Unless otherwise specified, the 
default values in the packages were used. 
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TABLE 1 
Summary of the data sets. 



Data set 



Reference # Examples # Features # classes 



Adenocarcinoma 


18 




76 


9868 


2 


Brain 


19 




42 


5597 


5 


Breast.2. class 


20 




77 


4869 


2 


Breast.3. class 


20 




95 


4869 


3 


Colon 


21 




62 


2000 


2 


Leukemia 


22 




38 


3051 


2 


Lymphoma 


23 




62 


4026 


3 


Nci60 


24 




61 


5244 


8 


Prostate 


25 




102 


6033 


2 


Srbct 


26 




63 


2308 


4 



adenocarcinoma 



method 

E|3 RRF 



method 

Ej3 GRFIF 



base coefficient 



importance coefficient 



(a) The number of features selected by RRF. A smaller A leads (b) The number of features selected by GRRF. A larger 7 leads 



to fewer features. 



to fewer features. 



leukemia lymphor 



• 9- 



prostate srbct ^ RRF 



-1 . "-^iis^ 



base coefficient 



(c) The error rates from RRF. 
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(d) The error rates from GRRF. The error rates are less sensi- 
tive to 7 than RRF 



Fig. 2. The performance of RRF with A e {0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.99 } and GRRF (guided RRF) with 7 e {0.005, 
0.01, 0.1, 0.2, 0.3, 0.4, 0.5}. 



We applied RF and a decision tree algorithm C4.5, 
respectively, to all the features, and the features selected 
by RRF, GRRF, varSelRF and LASSO logistic regression. 
The sizes of the feature sub sets and the cross-validation 
(CV) error rates from the classifiers were used to evaluate 
the feature selection methods. 

Work by [4] showed that the accuracy performance 
of RF is largely independent of the number of trees 
(between 1000 and 40000 trees), and mtry = s/P is often 
a reasonable choice. Therefore, we used rrfree=1000 and 
mtry = s/P for the classifier RF and the feature selection 
methods RRF, GRRF and varSelRF. Guided by initial 
experiments, randomly sampling data instances without 
replacement was used in RRF and GRRF. 



In the following we firstly show the advantages of 
GRRF over RRF, and then compare GRRF to varSelRF 
and LASSO logistic regression. 

5.1 Comparisons between RRF and GRRF 

We investigated the performance of RRF and GRRF with 
different parameter settings: A g {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 
0.7, 0.8, 0.9, 1} for RRF and 7 € {0.005,0.01,0.1,0.2,0.5} 
for GRRF (As discussed previously, we considered 7 
as the only parameter for GRRF). For each parameter 
setting, we used 10 rounds of 10-fold CV for evaluation. 
The procedure is described as follows. In each round, 
randomly partition a data set into 10 sub sets (folds) 
of equal (or approximately equal) sizes. Then conduct 
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(a) The variable importance scores calculated from (b) The variable importance scores calculated from 
the data with 50 instances. Only X2 has a clearly the data with 1000 instances. X\, X% and X3 have 
larger importance score than the noise features. clearly larger importance scores than others. 

Fig. 3. Plots of random forest importance scores calculated from the two data sets with a different number of instances. 
For both data sets, only the first three variables (labeled with"1 ", "2" and "3" respectively) are truly useful for predicting 
the class. 



10 experiments, in each of which use nine folds for 
training and the remaining one fold for testing. The 
average of the error rates from the 10 experiments is 
used as an estimate of the generalization error. Note for 
each experiment, RRF or GRRF is used to select a set of 
features based on the training folds. Then an RF is built 
on the feature sub set and applied to the testing fold to 
get the testing error rate. 

For each parameter setting of each method, let and 
Si, respectively, denote the average number of features 
selected and the average error rates of RF for the i th 
(i=l,2,...,10) round of CV 

The box plots of /, (i=l,2,...,10) for each parameter 
setting of RRF and GRRF, respectively, are shown in 
Figure 2(a)| and Figure 2(b)| The average number of 
features increases as A increases for RRF, and increases 
as 7 decreases for GRRF. The consistent trend illustrates 
that one can control the size of the feature sub set by 
adjusting the parameters. 

The box plots of ej (i=l,2,...,10) for each parameter 
setting of RRF and GRRF, respectively, are shown in 
Figure 2(c)| and Figure 2(d)| In general, the error rates 
tend to decrease as A increases for RRF. However, for 
most data sets, the accuracy performance of GRRF seems 
reasonably robust to the changes of 7. 

Furthermore, Figure H] plots e, versus /, for each 
parameter setting for RRF and GRRF, respectively. Note 
there are 10 points for each parameter setting in the 
figure. Compared to RRF, GRRF leads to, in general, 
lower error rates given a similar number of features. 
Also, the error rates from GRRF tend be more stable than 
those from RRF. Since GRRF has better performance than 
RRF, we omit the results of RRF in the following section. 

Furthermore, [14j mentioned the accuracy perfor- 
mance of RRF did not change dramatically as A changes 



for most of the data sets considered. However here 
RRF is quite sensitive to the changes of A. This may 
be because the data sets used in ||T4| and the data sets 
used here are very different: the number of features in 
the gene expression data sets considered here is much 
greater than the number of instances. To investigate the 
impact of this difference, we simulated two data sets 
with the same features, X\ : ■■■X\qqq, but with a different 
number of instances (50 and 1000,respectively). In the 
data sets, Xi is uniformly distributed between [—1,1]. Let 
Z be Xi + X 2 + X 3/ then the class variable is 1 if Z > 
and otherwise. Clearly only X\, X2 and X3 are useful 
features. Figure |3(a)| plots the RF importance scores for 
the data set with 50 instances, in which only X2 has a 
clearly larger importance score than the noise features. 
Figure |3(b)| plots the RF importance scores for the data 
set with 1000 instances. X\, X2 and X3 have clearly 
greater importance scores than others. In other words, 
the feature importance is more difficult to measure when 
the number of features is much greater than the number 
of instances. 



5.2 Comparisons among GRRF, varSelRF and 
LASSO logistic regression 

Now we compare GRRF to two well-known methods 
varSelRF and LASSO logistic regression by one round 
of 10-fold CV. We used the parameter selection proce- 
dure disused previously to automatically select 7 from 
{0.1, 0.2, 0.3, 0.4, 0.5} by 10-fold CV. The regularization 
parameter of LASSO logistic regression was also selected 
from {0.01, 0.02, 0.03, 0.05, 0.1, 0.2} by 10-fold CV 

In each experiment, GRRF, varSelRF or GRRF was 
used to select a set of features based on the training 
folds. Then an RF and C4.5, respectively, were built on 
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Fig. 4. The scatter plots of e. t versus fa {i = 1,2, 10) for each parameter setting for RRF and GRRF, respectively. 
Compared to RRF, GRRF leads to, in general, lower error rates given a similar number of features. Also, the error 
rates from GRRF tend be more stable than those from RRF. 



TABLE 2 

The number of original features (Orig) and the average number of features (from 10-fold CV) selected by different 
methods. All feature selection methods can select compact feature sub sets. 





Orig 


GRRF 


varSelRF 


LASSO 


adenocarcinoma 


9868 


20.4 


5.2 


4.3 


brain 


5597 


17.3 


28 


33.2 


breast.2. class 


4869 


25.9 


10 


12.2 


breast.3. class 


4869 


46.6 


9.1 


34.1 


colon 


2000 


16.8 


3.6 


8.2 


leukemia 


3051 


3.3 


2 


13.9 


lymphoma 


4026 


4.9 


59.3 


29.1 


nci 


5244 


64.1 


56.5 


68.2 


prostate 


6033 


10.1 


4.9 


15.8 


srbct 


2308 


12.8 


55.5 


34.3 



TABLE 3 

The error rates of RF and C4.5 applied to all the features (Orig) and the feature sub sets selected by GRRF, 
varSelRF and LASSO logistic regression, respectively. For the RF classifier, GRRF has better performance than 
varSelRF and LASSO logistic regression, and is competitive to Orig. For C4.5, GRRF has similar performance to 
varSelRF and has a slight advantage over LASSO logistic regression and Orig. 





Error rates from RF 


Error rates from C4.5 




GRRF 


varSelRF 


LASSO 


Orig 


GRRF 


varSelRF 


LASSO 


Orig 


adenocarcinoma 


0.182 


0.193 


0.196 


0.170 


0.284 


0.195 


0.220 


0.288 


brain 


0.225 


0.370 


0.440 


0.140 


0.505 


0.505 


0.510 


0.495 


breast. 2. class 


0.271 


0.407 


0.429 


0.393 


0.338 


0.370 


0.509 


0.457 


breast. 3. class 


0.369 


0.461 


0.431 


0.389 


0.389 


0.538 


0.487 


0.588 


colon 


0.245 


0.293 


0.355 


0.214 


0.352 


0.324 


0.455 


0.305 


leukemia 


0.000 


0.100 


0.175 


0.050 


0.025 


0.083 


0.200 


0.075 


lymphoma 


0.064 


0.048 


0.031 


0.014 


0.126 


0.150 


0.107 


0.181 


nci 


0.438 


0.471 


0.505 


0.355 


0.569 


0.569 


0.688 


0.571 


prostate 


0.059 


0.079 


0.313 


0.098 


0.188 


0.179 


0.354 


0.156 


srbct 


0.064 


0.014 


0.064 


0.031 


0.336 


0.300 


0.283 


0.331 


win/lose/tie 




2/8/0 


1/8/1 


6/4/0 




4/4/2 


3/7/0 


4/6/0 
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the feature sub set and applied to the testing fold to get 
the testing error rates. The average number of features 
selected by a feature selection method and the average 
error rates of the RF and C4.5 from the 10-fold CV can 
be calculated. 

The number of original features and the average num- 
ber of features selected by each feature selection method 
are shown in Table |2] Different feature selection methods 
may produce very different sub sets, e.g., GRRF selects 
fewer features than the other two for the lymphoma data 
set but selects more features for the adenocarcinoma data 
set. However, all the feature selection methods can select 
compact feature sub sets from the original feature sets. 

The average error rates of RF and C4.5 applied to 
all the features (Orig) and the sub sets selected by 
different feature selection methods are shown in Table |3j 
respectively. The win-lose-tie results of varSelRF, LASSO 
logistic regression and Orig compared to GRRF are also 
shown in the table. 

For the RF classifier, GRRF has a better performance 
than varSelRF (two wins and eight losses) and LASSO 
logistic regression (one win and nine losses). RF applied 
to all the features (Orig) has slightly better performance 
than GRRF (six wins and four losses). Note a feature 
selection method aims to eliminate those irrelevant and 
redundant features but potentially could also eliminate 
the features with a small amount of useful information, 
and therefore a strong classifier capable of capturing 
information from the data may slightly suffer from the 
lost information. Even though Orig has good accuracy 
performance, the number of the original features are 
much greater than the number of features from feature 
selection. Given a similar accuracy performance, feature 
selection may be still preferred in many cases. 

For the relatively weaker classifier, C4.5, the results are 
very different from the RF classifier. GRRF has a similar 
performance compared to varSelRF (four wins, four 
losses and two ties) and has an advantage over LASSO 
(three wins and seven losses). Interestingly, GRRF now 
is slightly better than Orig. This is consistent with the 
observation of (14| that a relatively weaker classifier may 
benefit from eliminating the noise features as it is more 
sensitive to the noise features and less capable of cap- 
turing information from the data. That may also partly 
explain why relatively weaker classifiers such as C4.5 
and the naive Bayes classifier have been commonly used 
for evaluating feature selection methods. Indeed, a weak 
classifier should be used for evaluation if that classifier 
is actually used for classification after feature selection. 
However, if a strong classifier is used for classification 
after feature selection, or the objective is to evaluate the 
information contained in the feature subsets, a strong 
classifier may be considered for evaluation ITHl . 

For either RF or C4.5, LASSO logistic regression leads 
to obviously worse performance compared to GRRF. 
The features selected by LASSO logistic regression do 
not necessarily fit the tree-based methods. Therefore we 
did another set of experiments using LASSO logistic 



regression as the classifier applied to the feature sub 
sets selected by LASSO logistic regression (results not 
shown here), the accuracy performance is competitive 
to the accuracy of RF applied to the feature sub sets 
selected by GRRF. When the number of instances are 
much smaller than the number of features, indeed many 
models such as a linear classifier can be good enough to 
describe the relationship between the predictor variables 
and the class; however, when the number of instances 
increases, the depths of trees in an RF are expected to 
increase, and the RF would be more capable of capturing 
interactions between the variables. 

6 Conclusion 

Here we propose a feature selection method called the 
guided RRF (GRRF), in which a preliminary random 
forest is used to generate the initial variable importance 
scores to guide the feature selection process of RRF. 
Experimental results show that GRRF is better than 
RRF and two well-known methods varSelRF and LASSO 
logistic regression in terms of the accuracy of RF and 
C4.5. 

7 Author's contributions 

HD designed and implemented the algorithm. HD and 
GR wrote the manuscript. 
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